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An  Evaluative  Study  of  the  Navy  Medical  Department's 
Patient  Classification  System  and  Staffing  Allocation 

CDR  Karen  A.  Rieder,  NC,  USN 
LCDR  Susan  S.  Jackson,  NC,  USNR-R 

1.  INTRODUCTION. 

a.  Purpose.  The  purpose  of  this  study  was  to  evaluate  the  validity  and 
reliablity  of  the  Workload  Management  System  (WMS)  for  Nursing  which  has  been 
under  development  since  1981.  In  addition,  the  perceptions  of  registered  nurses 
regarding  the  usefulness  of  the  system  as  a  management  tool  were  measured  using 
written  questionnaires.  This  technical  report  will  present  the  background 
information  for  the  study.  Including  the  hypotheses,  literature  review,  and 
instruments  utilized,  and  the  results  obtained  during  a  site  visit  to  the  first 
test  hospital.  Naval  Hospital  (NH)  Charleston.  A  final  report  will  compare  and 
analyze  the  findings  from  the  six  naval  hospitals  included  in  the  study. 

b.  Background.  The  Workload  Management  System  for  Nursing  was  developed  as 
part  of  a  three  year  research  project  which  examined  the  utility  of  a  patient 
classification  system  as  the  basis  for  nurse  staffing  decisions  within  the  Navy 
Medical  Department.  Initially  undertaken  at  the  request  of  the  Navy  Surgeon 
General,1  the  requirement  for  this  system  was  reinforced  by  a  mandate  from  the 
Joint  Commission  on  the  Accreditation  of  Hospitals.2  JCAH  Nursing  Standard  III 
states  that  departments  of  nursing  shall  define,  implement,  and  maintain  a 
system  for  determining  patient  requirements  for  nursing  care  on  the  basis  of 
demonstrated  patient  needs,  appropriate  nursing  Intervention,  and  priority  of 


As  a  prelide  to  development.  In  1980  a  survey  was  mailed  to  31  naval 

hospitals  to  ascertain  whether  or  not  patient  classification  systems  were  being 

used  to  determine  staffing  requirements.  Of  28  respondents,  twenty-one  or  75 

percent  of  the  hospitals  indicated  that  they  were  using  some  type  of  factor 
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evaluative  system  to  classify  patients.  However,  several  problems  were 
Identified  with  the  various  classification  tools.  First,  none  of  the  tools  had 
been  tested  for  reliability  to  determine  if  comparable  results  were  obtained  by 
different  users.  Second,  personnel  training  in  use  of  the  system  often  was 
Inadequate  which  resulted  In  low  user  satisfaction.  Third,  some  tools  were  very 
comprehensive  encompassing  several  Indicators  of  nursing  care  while  other  tools 
examined  only  a  few  Indicators.  Finally,  the  staffing  requirement  charts  for 
each  hospital  varied  significantly  according  to  both  the  number  and  the  mix  of 
nursing  personnel.  In  summary,  there  was  no  way  to  compare  workload  across 
naval  hospitals  due  to  the  lack  of  a  standardized  system. 

Based  upon  Information  from  the  survey,  five  patient  classification 
systems  were  selected  for  comparative  analysis.  A  panel  of  nurse  experts, 
each  experienced  In  using  one  system,  was  brought  together  to  evaluate  the  five 
systems.  During  the  study,  each  nurse  classified  patients  using  not  only  her 
own  hospital's  system  but  also  each  system  developed  by  the  other  four 
hospitals.  The  criteria  selected  for  evaluation  of  the  systems  were:  the  scope 
of  use.  Inter-rater  reliability,  ease  of  use,  and  user  acceptance. 

Using  the  above  criteria,  one  system  was  judged  to  be  superior  to  the 
others.  The  Jacksonville  System  was  found  to  have  the  highest  inter-rater 
reliability  (89.5  percent  In  1,222  items),  greatest  ease  of  use,  and  the  highest 
level  of  acceptance.4  In  addition,  this  system  was  more  discriminating  in  that 
patients  were  grouped  into  five  categories  of  nursing  care,  rather  than  three  or 
four.  Finally,  the  staffing  requirement  charts  used  in  the  Jacksonville  system 


were  found  to  be  at  the  median  level  for  number  and  mix  of  personnel  among  the 
five  systems.  Guided  by  the  findings  of  this  study,  the  Director  of  the  Navy 
Nurse  Corps  selected  the  Jacksonville  patient  classification  system  for 
standardization  at  all  naval  hospitals. 

In  January  1982  a  refined  workload  management  system  was  pre-tested  at 
selected  naval  regional  medical  centers  and  distributed  to  the  34  hospitals 
during  a  pilot  test.  This  system  was  developed  for  use  on  medical-surgical, 
neuro-surglcal,  orthopedics,  post-partum,  and  multi-service  wards.  After  months 
of  trial  testing,  several  problems  were  detected.  Foremost  was  low  user 
acceptance  because  the  philosophy  underlying  the  system  was  based  upon  the 
concept  of  minimal  staffing  to  deliver  patient  care.  The  system  had  been 
designed  to  Identify  the  minimum  number  of  staff  necessary  to  give  "safe"  versus 
an  optimal  number  of  staff  to  give  "quality"  care.  Nurses  In  the  field 
perceived  that  Washington  did  not  understand  the  staffing  requirements  for  good 
care  and  subsequently  padded  the  scores  to  increase  the  staffing  numbers.  Other 
problems  Identified  included:  misinterpretation  of  critical  indicators  of  care, 
complaints  about  weights  (minutes  of  time)  attached  to  each  critical  indicator, 
and  questions  concerning  applicability  of  the  system  to  special  care  areas. 

Based  upon  feedback  from  the  pilot  test,  the  following  changes  were  made: 

1)  The  philosophy  upon  which  the  workload  management  system  Is  based 
was  changed  from  minimal  staff  to  optimal  staff  for  delivery  of 
quality  care. 

2)  The  point  value  for  each  critical  Indicator  was  standardized  using 
values  obtained  from  mathematical  time  and  motion  studies. 

3)  The  critical  indicator  list  of  direct  care  activities  was  streamlined 
to  include  only  critical  Indicators  having  the  greatest  impact  on 
nursing  time.  In  addition,  new  critical  indicators  which  pertained 
to  specialty  areas  were  added. 

4)  The  categories  of  care  were  expanded  from  five  to  six  levels  so  that 

the  system  could  be  used  in  critical  care,  nursery,  and  pediatric  settings. 


5)  The  nursing  care  hour  requirement  charts  for  determining  the  number 

and  mix  of  nursing  personnel  were  revised  to  account  for: 

(a)  direct  and  indirect  nursing  care  activities, 

(b)  teaching  versus  non-teaching  hospital  variations,  and 

(c)  type  of  unit  variations:  open,  semi-private,  nursery,  or 
light  care. 

To  standardize  the  point  value  for  each  critical  indicator  the  Navy 
incorporated  the  results  obtained  in  a  Nursing  Care  Hour  Standards  Study  (1981) 
completed  by  the  Army.  LTCOL  Susan  Sherrod,  a  nurse  researcher  at  the  Health 
Care  Studies  Division  of  the  Academy  of  Health  Sciences,  Fort  Sam  Houston, 

Texas,  conducted  a  four-year  time  and  motion  study  during  which  37,000 
observations  were  made  to  derive  mean  times  for  357  direct  nursing  care 
activities  at  nine  hospitals  for  all  levels  of  nursing  staff.5  The  time 
measurements  were  utilized  to  determine  the  minimal  essential  mean  tasking  time 
for  all  specified  direct  nursing  care  activities.  In  addition,  a  methodology 
for  determining  care  provider  numbers  and  mix  for  various  specialty  areas  was 
developed. 

In  adopting  the  time  values,  an  assumption  was  made  that  direct  nursing 
care  time  for  specific  activities  in  the  Navy  was  the  same  as  direct  care  time 
in  the  Army.  This  assumption  was  tested  by  actually  timing  certain  selected 
activities  and  found  to  be  valid.  The  Sherrod  study  has  been  cited  as  having 
the  most  comprehensive  and  best  documented  task  list  to  be  found  anywhere  in 
nursing  literature.6  For  these  reasons,  the  Navy  has  a  high  level  of  confidence 
in  the  point  values  assigned  to  each  critical  indicator  of  direct  care. 

After  incorporating  all  of  the  recommendations,  a  revised  Workload 
Management  System  was  implemented  in  January  1983.  Although  feedback  indicated 
greater  user  satisfaction,  there  were  still  variances  in  how  patients  in 
homogeneous  groups  i.e.,  newborns,  were  classified.  It  was  hypothesized  that 


the  classification  tool  was  not  being  used  reliably  due  to  confusion  about  the 
definitions  of  critical  Indicator  terms  and  insufficient  guidelines  for 
Implementing  the  system. 

To  validate  these  Impressions,  on-site  studies  were  conducted  at  two 
large  naval  hospitals  during  Aprll-May  1983.  The  purpose  of  the  visits  was  to 
assess  the  reliability  and  validity  of  this  revised  Workload  Management  System. 
Findings  from  the  visits  revealed  Inconsistencies  In  patient  classification  and 
confusion  regarding  proper  Implementation  and  monitoring  of  the  system.7 

Corrective  action  was  taken  to  further  revise  the  critical  Indicator 
list,  to  operationally  define  the  critical  indicator  terms,  and  to  develop  a 
-andardlzed  teaching  workbook.  In  July  1983,  a  three-day  workshop  was  held  for 
the  Workload  Management  Coordinators  from  all  CONUS  naval  hospitals  to  discuss 
implementation  problems,  to  disseminate  materials,  and  to  devise  a  mechanism  for 
monitoring  system  reliability.  As  a  result  of  the  workshop,  a  revised  Critical 
Indicator  Instrument  for  classifying  patients  was  developed.  For  the  first 
time,  a  Workload  Management  System  Educational  Workbook  to  be  used  for 
personalized  instruction  or  for  inservice  teaching  was  written  and  distributed 
to  all  naval  hospitals.  In  this  way  the  Workload  Management  System  process  and 
terminology  were  standardized  In  all  hospitals. 

By  October  1983,  most  CONUS  hospitals  had  Incorporated  the  new 
Critical  Indicator  Instrument  for  classifying  patients,  had  conducted  training 
classes  using  the  workbook  for  all  nurses  Involved  with  the  WMS,  and  had 
Instituted  monthly  inter-rater  reliability  testing  on  nursing  units.  Out  of 
CONUS  hospitals  Implemented  the  new  changes  by  January  1984.  Figure  1  describes 
how  the  system  is  being  utilized  by  all  naval  hospitals. 

To  complete  the  Workload  Management  System  Project,  an  evaluative 
study  was  begun  in  February  1984  to  assess  Its  validity  and  reliability.  The 


DYNAMICS  OP  THE  WORKLOAD  MANAGEMENT  SYSTEM 


FIGURE  1 


The  flow  chart  above  delineates  how  the  Workload  Management  System  operates. 
The  process  begins  with  the  classification  of  patients  Into  categories  of  care. 
The  hours  of  nursing  care  required  and  the  recommended  number  and  mix  of 
personnel  needed  to  meet  these  requirements  are  then  calculated  based  on  the 
number  of  patients  In  each  category.  The  actual  number  and  mix  of  personnel 
assigned  Is  then  compared  with  the  recommended  staffing  to  determine  If  staffing 
levels  are  above,  below,  or  within  the  recommendations.  If  staffing  levels  for 
the  workload  to  be  accomplished  differ  from  recommended  levels,  staffing  can  be 
adjusted  to  balance  the  variation.  The  number  and  mix  of  nursing  personnel 
available  to  provide  patient  care  will  significantly  Impact  on  the  quality  of 
care  actually  delivered. 
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preliminary  results  described  In  this  report  are  based  upon  an  on-site  visit  to 
Naval  Hospital,  Charleston,  South  Carolina.  Over  the  next  six  months, 
additional  on-site  visits  will  be  made  to  five  other  CONUS  hospitals  purposely 
selected  based  upon  size.  Inter-rater  reliability  was  tested  by  comparing 
scores  awarded  by  a  nurse  expert  with  scores  assigned  by  the  nurse  classifying 
patients  on  the  same  nursing  units.  Other  data  was  collected  by  administering 
questionnaires  which  measure  charge  nurses'  perceptions  of  staffing  adequacy  and 
staff  nurses'  perceptions  of  direct  and  indirect  nursing  care  given  under 
various  staffing  patterns. 

Throughout  the  Workload  Management  System  Project,  close  collaboration 
has  been  maintained  with  the  Army  Nursing  Research  Department,  Walter  Reed  Army 
Medical  Center,  Washington,  DC.  To  check  for  construct  validity,  the  Army  has 
field  tested  the  WMS  at  Army  Military  Treatment  Facilities.  In  addition, 
the  Army  has  recently  completed  a  work  sampling  study  of  indirect  nursing  care 
at  nine  hospitals.8  Results  from  this  study  will  be  valuable  In  determining 
whether  or  not  the  Navy  Nursing  Care  Hour  Requirement  Charts  need  to  be  revised. 

c.  Merits. 

The  benefits  to  the  Navy  of  a  reliable  Workload  Management  System  are 
multiple.  At  the  local  hospital  level,  the  information  can  be  used  to  determine 
workload,  to  compare  workload  on  different  units,  to  adjust  staff  assignments, 
and  to  justify  requests  for  additional  numbers  and  skill  level  of  nursing 
personnel.  The  Naval  Medical  Command  could  use  the  information  for  identifying 
the  number  of  beds  and  units  required  in  a  new  facility  based  on  historical 
workload,  for  identifying  seasonal  trends  in  patient  requirements  at  various 
facilities,  and  for  determining  the  number  and  types  of  nursing  personnel 
required  for  care  of  specified  groups  of  patients  under  wartime  scenarios.  The 
Navy  Military  Personnel  Command  could  utilize  the  data  for  equitably 


distributing  nursing  billets  among  hospitals.  This  data  could  also  be 
valuable  as  an  addendum  to  the  Uniform  Staffing  Methodology  and  Uniform  Chart  of 
Accounts  materials 'from  which  measurement  and  justification  of  health  care  costs 
are  determined. 

2.  REVIEW  OF  LITERATURE. 

One  of  the  most  difficult  problems  facing  directors  of  nursing  service 
Is  the  determination  of  nurse  staff  requirements.  Traditionally,  staffing  needs 
were  based  upon  dally  Inpatient  census,  and  staffing  standards  were  set  on  a 
ratio  of  hours  of  care  per  24  hour  patient  day.  For  example,  3.5  hours  of  care 
per  patient  day  may  be  used  with  staffing  levels  generally  set  high  to  ensure 
adequate  coverage  during  peak  loads.9  Another  major  problem  was  the  assumption 
that  nursing  care  requirements  would  be  the  same  for  every  patient.  The  result 
was  a  continuous  Inbalance  of  workload.  Although  supervisors  made  daily 
staffing  reassignments  based  upon  their  judgment  and  expertise,  these  changes 
often  were  viewed  as  subjective  and  frequently  resulted  in  charges  of  unfairness 
or  favoritism.  Therefore,  studies  were  undertaken  to  find  a  more  objective 
method  for  determining  staffing  requirements. 

The  earliest  studies  that  could  be  found  were  undertaken  by  the  National 
League  of  Nursing  Education.  These  evolved  from  concern  regarding  the 
appropriate  tasks  and  numbers  of  patients  assigned  per  level  of  nursing 
personnel,  the  most  efficient  combination  of  staff,  and  the  minimum  number  of 
staff  to  hire.  In  a  1947  study  an  Initial  attempt  was  made  to  base  staffing 
numbers  upon  nursing  care  requirements  of  patients  by  rating  pediatric  patients 
on  a  three-point  Intensity  scale  using  four  factors:  degree  of  illness, 
activity,  adjustment,  and  the  number  and  complexity  of  procedures  and  treatments. 
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Wright  In  1954  was  among  the  first  to  utilize  work  study  techniques  developed  by 

Industrial  engineers  to  quantify  the  amount  of  time  required  for  specific 

nursing  interventions.11  A  study  which  further  demonstrated  the  need  to  match 

staffing  to  patient  workload  was  done  by  Aydelotte  and  Tener  in  1960.  They 

found  that  when  the  numbers  of  nursing  personnel  on  a  unit  were  increased 

without  a  parallel  Increase  in  patient  load,  the  staff  did  not  redistribute 
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their  time  to  allow  for  more  direct  care  to  patients.  This  finding  has 
ramifications  for  "overstaff Ing"  or  placing  more  staff  than  is  needed  on  a 
rfursing  unit. 

One  widely  used  technique  for  determining  workload  is  a  patient  classification 

system  (PCS).  Under  this  method,  staffing  requirements  are  based  upon  the 

acuity  levels  of  patients.  A  patient  classification  system  refers  to  the 

"identification  and  classification  of  patients  into  care  groups  or  categories, 

and  to  the  quantification  of  these  categories  as  a  measure  of  the  nursing  effort 
13 

required".  Abdellah  and  Levine  have  Identified  two  types  of  patient 

classification  systems.  The  first  type,  "prototype  evaluation"  matches  patients 

to  a  standard  profile  or  stereotype  for  each  category.  The  second  and  most 

common  type,  "factor  evaluation",  utilizes  critical  Indicators  or  descriptors  of 

direct  care  requirements  to  Individually  rate  the  care  required  by  each  patient. 

Each  factor  earns  a  score  which  Is  then  summed,  and  the  total  score  determines  a 

K 

patient's  category. 

The  studies  having  the  greatest  Impact  on  the  modern  patient  classification 
system  were  conducted  by  R.J.  Connor  In  the  late  50's  and  early  60's  at  Johns 
Hopkins  Hospital.  He  used  work  measurement  techniques  to  identify  elements  of 
direct  patient  care.  Patients  were  classified  into  three  categories  (self  care, 
partial  care,  and  total  care)  depending  upon  their  nursing  care  requirements. 


9 


For  each  category,  Connor  developed  an  average  number  of  nursing  care  hours  per 
patient  day  which  were  based  upon  Industrial  time  and  motion  measurements!5  For 
the  first  time  a  patient  classification  system  could  be  converted  Into  a 
quantitative  measurement  of  nursing  workload.  Although  a  milestone  In 
objectively  determining  staffing  needs,  the  research  was  limited  because  It  only 
examined  direct  care  requirements  and  not  Instructional,  observational,  and 
emotional  needs  of  the  patient  nor  nursing  administrative  duties.  At  this  point 
In  time,  nursing  was  primarily  seen  as  a  task  oriented  occupation.  Young  (1962) 
was  the  first  to  Include  Indirect  nursing  care.  He  defined  Indirect  care  as 
"all  activities  done  away  from  the  patient  In  preparation  for  completion  of 
care,  as  well  as  those  activities  directed  toward  general  unit  management". 16 

Within  the  past  twenty  years,  patient  classification  systems  have  not 
changed  drastically  but  rather  have  been  refined  and  tested  for  validity  and 
reliability.  Several  commercial  Instruments  have  emerged  such  as  CASH,  TISS, 
MEDICUS,  and  GRASP  which  have  been  modified  for  use  In  multiple  types  of 
hospitals.  In  an  attempt  to  provide  nurse  staff  planners  with  a  method  for 
studying  and  estimating  staffing  needs,  the  U.S.  Public  Health  Service,  Olvlsion 
of  Nursing,  conducted  an  eight  year  study  at  San  Joaquin  General  Hospital, 
Stockton,  California.  The  study  resulted  In  Identifying  procedures  for  work 
sampling,  direct  care  sampling,  patient  classification,  and  measurements  of 

perception  of  staff  adequacy.  The  study  also  documented  the  relationship 
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between  patient  classification  and  staff  needs.  As  a  result,  a  standardized 

manual  for  evaluating  a  hospital's  staffing  requirements  was  developed.  This 

manual  was  updated  In  1983  to  Incorporate  new  measurement  techniques  after  a 
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fo11ow>up  study  of  the  San  Joaquin  methodologies  was  conducted. 

Despite  all  of  the  development  and  research  on  patient  classification 


systems  over  the  past  thirty  years,  significant  shortcomings  still  exist. 

Vaughan  and  MacLeod  (1980)  Identify  some  of  the  problems  as  follows: 

1)  The  studies  generate  far  more  data  than  are  needed  for  practical 
application. 

2)  The  scheme  for  classifying  patients  varies  among  hospitals. 

3)  Most  classification  schemes  leave  gray  areas  between  classes 
within  a  hospital. 

4)  The  workload  analysis  systems  are  generally  too  tailored  to  one 
Institution. 

5)  Most  systems  cannot  be  easily  updated  when  changes  occur  In  methods, 
physical  unit  layout,  equipment,  or  redistribution  of  certain 
activities  to  other  departments. 

6)  Most  systems  have  no  means  of  verifying  that  the  operational  classifying 
of  patients  Is  being  done  accurately. 

Although  Vaughan  and  McLeod  support  the  Idea  of  a  standardized  tool  so  that 

comparative  analysis  of  one  hospital's  nursing  staff  efficiency  can  be  made  to 

other  hospitals  or  national  norm,  other  researchers  disagree.  Glovannettl  (1979) 

stated  that  the  qualification  coefficients  are  not  transferable.  She  asserted: 

"Average  care  times  or  standard  times  are  not  necessarily  the 
same  from  Institution  to  Institution  or  even  from  one  nursing 
unit  to  another.  A  multitude  of  factors  affect  the  determination 
of  care  times,  such  as  type  of  nursing  organization,  physical 
facilities  design,  treatment  modalities,  physician  practices, 
availability  of  nursing  skills,  and  attitudes  of  treatment  personnel."  20 

Therefore,  hospitals  Interested  In  using  a  patient  classification  system 

need  to  determine  their  own  quantification  or  index  of  workload.  This 

requires  a  modification  of  commercial  systems  to  meet  the  needs  of  each 

hospital . 

More  recently,  researchers  have  attempted  to  demonstrate  a  relationship 
between  the  nursing  care  hours  per  patient  generated  by  a  workload  management 
system  and  the  quality  of  patient  care.  Glovannetti  (1979)  believes  that: 


"Patient  classification  systems  can  be  extremely  effective 
in  matching  workload  generated  by  patients  to  numbers  of 
nursing  personnel.  There  is  no  evidence,  however,  that 
the  effective  utilization  of  nursing  personnel  hours  has 
any  direct  relationship  to  the  quality  of  care."21 

One  problem  in  determining  quality  is  the  overlap  of  functions  performed 

by  various  levels  of  nursing  personnel.  A  study  by  the  Allied  Health 

Professionals  at  UCLA  found  that  60  percent  of  all  nursing  functions  were 
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performed  by  all  levels  of  nursing  personnel  -  aide,  LPN,  and  RN.  Therefore,  a 
variety  of  personnel  configurations  may  be  used  to  meet  patient  care  needs. 

To  specifically  look  at  the  quality  of  nursing  care  in  relation  to 
workload  in  the  Navy  Medical  Department,  Montgomery  and  Kelly  utilized  the 
Rush-Medicus  quality  instrument.  Phase  I  of  their  research  determined  which  of 
the  250  questions  were  indicative  of  quality  care  given  on  a  particular  shift. 

Of  these  questions,  140  were  found  to  be  relevant.  In  Phase  II,  staffing 
structures  were  identified.  The  analysis  of  Phase  II  data  "failed  to 
demonstrate  a  practical  significant  relationship  between  the  quality  of  nursing 
care  at  naval  hospitals  and  nursing  personnel  workload  as  measured  either  by  the 
average  daily  difficulty  of  assignments  or  by  the  average  hours  per  person 
required  by  staff  members  to  care  for  patients."  23 A  third  phase  of  the  research 

consisted  of  asking  nursing  supervisory  personnel  to  select  those  quality 
questions  on  the  questionnaire  they  believed  most  sensitive  to  changes  in 
workload.  Twenty-five  questions  were  selected.  Although  there  was  a  stronger 
correlation  to  workload  changes,  the  results  again  were  not  significant.  The 

authors  concluded  that  the  most  vitally  needed  component  for  future  quality 
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workload  studies  is  a  valid  quality  monitoring  methodology.  Unfortunately,  a 
further  review  of  the  literature  has  failed  to  identify  such  an  instrument. 

Williams  and  Murphy  (1979)  attempted  to  measure  staffing  adequacy, 
patient  care  services  provided  under  various  staffing  conditions,  and  charge 


nurses'  subjective  judgment  of  both  of  these  elements.  The  results  showed  that 

charge  nurses'  judgments  of  less  than  satisfactory  staffing  were  strongly 

related  to  perceived  declines  in  their  ability  to  provide  direct  care  services. 

They  also  found  that  when  staffing  was  judged  as  Inadequate,  priority  was  given 

to  activities  of  the  medical  plan  while  communications  and  observation  of 
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patients  suffered.  These  results  cannot  be  generalized  since  the  validity  and 
reliability  of  the  tools  to  measure  these  perceptions  were  not  addressed  In 
their  study. 

In  conclusion,  patient  classification  systems  appear  to  be  useful  models 
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for  evaluating  staffing  adequacy  and  for  determining  long  term  staffing 

requirements.  Yet,  their  utility  Is  predicated  upon  the  close  attention  given 

to  developing  various  elements  of  the  system.  The  critical  Indicators  which 

best  Identify  the  appropriate  category  of  care  must  be  based  upon  observational 

studies,  although  there  is  considerable  debate  about  the  number  and  scope  of  the 

critical  indicators.  Glovannetti  (1979)  does  not  believe  that  the  patient 

classification  tool  must  provide  a  comprehensive  assessment  of  patient  needs. 

She  thinks  that  It  Is  sufficient  to  Identify  direct  care  requirements  with 

minimal  attention  given  to  psychosocial  and  teaching  needs.  In  fact,  there  are 
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some  tools  with  only  four  Indicators  that  have  been  found  to  be  valid.  Other 
model  developers  believe  only  a  system  that  is  Inclusive  of  all  the  requirements 
of  patient  care  is  sufficient.  This  Is  based  on  the  assumption  that  staff 
nurses  using  the  Instrument  will  not  accept  Its  credibility  if  direct  and 
Indirect  care  activities  are  not  Included.  If  not  perceived  as  comprehensive, 
the  instrument  may  not  be  used  reliably. 

Reliability  and  validity  are  key  elements  in  any  patient  classification 
system.  In  order  to  ensure  that  nurses  classify  the  same  patient  Into  the  same 


category  of  care,  a  reliability  monitoring  system  requiring  an  80  percent 
Inter-rater  reliability  level  must  be  established.  Validity  Is  a  more  difficult 

element  to  demonstrate  and  refers  to  the  extent  to  which  an  Instrument  actually 

* 

measures  what  It  purports  to  measure.  The  content  validity  of  most  patient 
classification  Instruments  has  been  determined  by  a  panel  of  nurse  experts. 
Construct  validity  has  seldom  been  verified,  and  Giovannettl  states  that  none  of 

the  present  Instruments  have  demonstated  validity  in  terms  of  actual  patient 
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needs. 

3.  DEFINITION  OF  TERMS. 

Workload  Management  System  for  Nurses:  a  systematic  process  for  determining 
staffing  requirements  based  upon  Identified  patient  care  needs.  The  system 
Includes  a  patient  classification  tool  and  a  staffing  methodology.  The  patient 
classification  Instrument  is  of  factor  evaluative  design  and  requires  that  a 
registered  nurse  assess  ten  factors  related  to  direct  patient  care  and  assign  a 
score  to  each  factor.  The  assessment  consists  of  both  retrospective  and 
prospective  components;  that  is,  assessment  of  care  required  during  the  day 
shift  Is  used  to  predict  care  requirements  for  the  next  24  hours.  The  weighted 
factor  scores  are  suamed  resulting  In  the  patient  being  classified  Into  one  of 
six  discrete  categories.  The  staffing  methodology  Is  used  for  determining  the 
actual  nursing  care  hour  requirements  for  a  specified  group  of  patients,  and  the 
numbers  and  mix  of  personnel  recommended  to  provide  quality  care.  This  system 
Incorporates  both  direct  and  indirect  care  time. 

Patient  Classification:  the  grouping  of  patients  according  to  an 
assessment  of  their  nursing  care  requirements  over  a  specified  period  of  time. 
Critical  Indicators;  those  nursing  activities  on  the  patient  classification 


instrument  that  have  the  greatest  impact  on  direct  care  time. 


Factors:  a  group  of  critical  indicators  that  cover  one  specific  domain 
of  activities.  They  include  nine  areas:  vital  signs,  monitoring,  activities  of 
daily  living,  feeding,  treatments/procedures/medications,  respiratory  therapy, 

IV  therapy,  teaching/emotional  support,  and  continuous  care. 

Points:  the  values  assigned  to  each  specific  critical  Indicator  based  upon 
documented  time  and  motion  studies.  Each  point  is  equal  to  7  1/2  minutes  of 
direct  nursing  care  time. 

Category:  the  representative  grouping  of  patients  according  to  their 
nursing  care  requirements.  The  Workload  Management  System  consists  of  six 
categories.  A  category  I  patient  requires  minimal  care  whereas  a  category  VI 
patient  requires  intensive  care. 

Direct  Nursing  Care  Time:  the  activities  that  take  place  in  the  presence  of 
the  patient  and/or  family.  These  activities  are  observable,  behavioral,  and 
include  the  following:  placement  of  equipment  at  bedside,  explanation  of 
procedure  to  patient,  preparation  of  patient,  performance  of  treatment,  removal 
of  equipment  from  area,  recording  of  treatment,  assessment/observation  of 
patient  response,  and  teaching.  These  activities  account  for  35  to  45 
percent  of  the  total  nursing  care  time. 

Indirect  Nursing  Care  Time:  those  activities,  conditions,  and  circumstances 
that  necessitate  time  over  and  above  direct  care.  To  address  these  factors, 
indirect  care  time  and  special  allowances  have  been  incorporated  into  the 
nursing  care  hour  requirements  for  each  of  the  six  patient  care  categories. 
Indirect  care  time  accounts  for  approximately  55  to  65  percent  of  the  total 
nursing  care  time  and  Is  subdivided  into  the  following  four  categories: 
indirect  care  -  30*,  unpredicted  needs  -  15X,  teaching  hospital  allowance  -  10*, 
and  semi -private  room  allowance  -  20*. 
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Nursing  Care  Hour  Requirements:  the  hours  of  nursing  care  time  required  for 
each  category  of  patient  based  upon  an  assessment  of  their  direct  and  Indirect 
nursing  care  requirements.  This  Is  operationalized  via  six  pre-calculated 
nursing  care  hour  requirement  charts  which  Incorporate  two  factors:  type  of 
unit  (open,  semi-private  room,  nursery,  or  light  care)  and  type  of  facility 
(teaching  vs.  non-teaching  hospitals). 

Personnel  Requirements:  the  number  and  mix  of  RNs  and  NRNs  required  to  care 
for  the  patient  workload  on  a  unit.  This  is  operationalized  via  two  charts: 
acute  care  and  intensive  care.  The  acute  care  chart  allocates  a  4 OX  RN  to  60X 
NRN  personnel  mix  and  distributes  45X  of  staff  to  the  day  shift,  35X  to  the 
evening  shift,  and  20X  to  the  night  shift.  In  contrast,  the  Intensive  care 
chart  utilizes  a  60X  RN  to  40X  NRN  personnel  mix  which  is  evenly  distributed 
across  all  shifts. 

RN:  a  professional  Registered  Nurse  who  has  satisfactorily  completed 
an  orientation  program  to  the  hospital. 

NRN:  Nursing  Service  personnel  other  than  RNs  who  have  satisfactorily 
completed  an  orientation  program  to  the  hospital.  This  includes  corpsmen,  LPNs, 
and  medical  ward  clerks. 

Workload  Index:  the  ratio  of  recommended  staffing  levels  to  assigned 
staffing  levels  as  determined  by  the  Workload  Management  System.  An  Index  of 
less  than  1.0  Indicates  assigned  staff  exceeds  recommended  staff,  reflecting 
overstaffing.  An  Index  of  greater  than  1.0  reflects  understaff Ing;  that  is, 
assigned  staff  does  not  meet  the  recommended  staffing  level. 

Trained  Rater:  a  nurse  who  has  undergone  standardized  training  in  use  of 


the  patient  classification  instrument. 


Inter-Rater  Reliability:  level  of  agreement  (in  factors  and  in 
categories)  achieved  when  two  trained  raters  independently  assess  a  group  of 
patients  on  a  specified  unit  during  the  same  time  period  using  the  patient 
classification  instrument.  An  80%  level  is  desirable. 

Intra-Service  Rel_1ab11U^:  level  of  agreement  in  factors  and  in 
categories  within  each  clinical  service  (medicine,  surgery,  orthopedics, 
nursery,  ICU,  CCU,  post-partum)  when  trained  raters  Independently  classify 
patients  using  the  patient  classification  Instrument.  An  80%  level  is 
desirable. 

4.  HYPOTHESES 

In  order  to  determine  whether  or  not  the  Workload  Management  System  (WMS)  Is 
a  valid  and  reliable  Instrument  for  quantifying  patient  care  needs  and  for 
establishing  manpower  requirements,  the  following  hypotheses  were  tested: 

a.  The  WMS  will  have  high  inter-rater  reliability  (percent  agreement  >  80%) 
plus  Kappa  significant  at  p  <  .01)  for  patient  category  using  the  classification 
at  each  test  facility. 

b.  There  will  be  high  inter-rater  reliability  (Intra-class  correlation 
significant  at  p  <  .01)  among  a  majority  of  the  critical  factors  in  the  patient 
classification  instrument. 

c.  Nurses  will  express  greater  acceptability/satisfaction  with  the  WMS  than 
dissatisfaction. 

d.  There  will  be  a  positive  correlation  between  nursing  staff  numbers  and 
mix  (as  determined  by  the  WMS)  and  charge  nurses'  perceptions  of  staffing 
adequacy. 

e.  There  will  be  a  positive  correlation  between  staff  nurses'  perceptions  of 
the  quality  of  direct  and  indirect  care  provided  and  the  Workload  Index  which 
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Indicates  under  (-)  or  over  (+)  staffing. 

5.  METHODOLOGY 

a.  Overview.  Six  study  sites  were  selected  to  provide  a  representative  mix 
of  CONUS  naval  hospitals  by  size,  geographic  location,  nursing  unit 
configuration,  and  mission  to  population  served.  Additional  criteria  Included 
the  availability  of  up-to-date  monthly  nurse  staffing  summary  Information  and 
reports  of  hospital  Inter-rater  reliability  testing  for  a  minimum  of  four 
months.  The  Directors  of  Nursing  Service  at  hospitals  which  met  the  criteria 
were  consulted  regarding  their  willingness  to  participate  in  the  study.  An 
official  letter  from  the  Commanding  Officer,  Naval  School  of  Health  Sciences, 
Bethesda,  MO,  was  sent  to  the  Commanding  Officer  of  each  selected  hospital  to 
request  formal  permission  to  make  an  on-site  visit. 

This  technical  report  will  present  and  discuss  the  findings  of  the  study  at 
the  first  test  site.  Naval  Hospital  Charleston.  The  procedure  outlined  was 
followed  at  all  subsequent  study  sites. 

b.  Procedure. 

(1)  Prior  to  making  the  first  on-site  visit,  the  Nurse  Expert  established 
her  Inter-rater  reliability  by  classifying  patients  at  Walter  Reed  Army  Medical 
Center  with  the  Critical  Indicators  Instrument.  The  nurse  expert's  scores  were 
compared  to  scores  obtained  by  an  Army  Nurse  Expert  for  the  same  patients.  A  90 
percent  Inter-rater  reliability  level  was  obtained  on  factor  comparisons  and  100 
percent  for  categories. 

(2)  The  data  collection  instruments  (see  Appendices  A-E)  were  developed 
jointly  with  the  Army  Nursing  Research  Department.  Several  tools  were  updated 
versions  of  those  used  in  the  San  Joaquin  General  Hospital  study.  Prior  to  the 


on-site  visit,  the  questionnaires  were  reviewed  by  nurse  experts  and  field 
tested  by  the  Army  and  Navy  for  content  validity.  The  Workload  Management 
Sumnary  Sheet  (see  Appendix  F),  currently  being  completed  by  all  naval 
hospitals,  was  used  to  obtain  specific  information  about  the  number  and  mix  of 
staff  assigned  by  shift  to  each  nursing  unit  studied.  The  information  from  this 
sheet  was  translated  into  a  Workload  Index  modified  from  a  study  done  by  Dale 
and  Mable  in  1983. 28 

(3)  From  the  daily  Workload  Management  Summary  Sheets  mailed  monthly  by 
the  WMS  Coordinator  at  NH  Charleston  to  the  Research  Department,  Naval  School  of 
Health  Sciences,  a  composite  computer  output  report  was  prepared.  This  output 
report  sumnarlzed  monthly  staffing  figures  and  reflected  which  nursing  units  had 
problems  with  overstaffing  and  understaffing.  The  data  included  all  submitted 
reports  since  October  1983.  By  examining  the  figures  and  graphs,  the  principal 
investigator  was  able  to  pre-select  particular  nursing  units  with  staffing 
variances  within  the  hospital.  This  served  as  a  basis  for  determining  which 
nursing  units  would  be  selected  for  reliability  testing  during  the  on-site 
visit.  In  addition,  the  computer  reports  were  made  available  to  the  Director  of 
Nursing  Service. 

(4)  The  Director  of  Nursing  Service  at  NH  Charleston  provided  the 
principal  investigator  with  the  following:  a)  the  name  of  the  Workload 
Management  System  Project  Officer;  b)  verification  of  all  inpatient  nursing 
units  broken  down  by  clinical  service,  type  of  room,  and  number  of  shifts  within 
24  hours;  and  c)  administrative  space  and  support  during  the  on-site  visit.  In 
addition,  the  researcher  offered  to  present  a  one  hour  briefing  on  the  system  to 
all  professional  nursing  personnel  and  a  separate  briefing  to  the  Commanding 
Officer  and  selected  administrative  staff. 


(5)  Reliability  testing  of  the  Patient  Classification  Critical  Indicators 
Instrument  (see  Appendix  G)  was  conducted  by  the  expert  classifier  on  the 
following  units:  ICU,  CCU,  Pediatrics,  Nursery,  and  pre-selected 

medical -surgical  units  based  upon  historical  workload  reports.  The  number  of 

patients  classified  represented  at  least  20  percent  of  the  Inpatient  census. 

Patients  were  randomly  selected  from  each  category  of  patients  on  a  nursing 

unit.  A  maximum  of  five  patients  per  category  were  classified.  Reliability 

testing  was  conducted  for  one  day  on  each  unit  between  the  hours  of  1130  -  1500. 

The  patient  profiles,  charts,  and  flowsheets  were  reviewed  for  documentation 

evidence  during  testing,  and  scores  obtained  by  the  nurse  expert  were  compared 

to  scores  assigned  by  the  charge  nurse  or  the  daily  Patient  Classification 

Worksheet  (see  Appendix  H).  An  agreement  level  of  80  percent  was  set  as  the 

minimal  acceptable  criteria.  Agreement  was  sought  between  categories,  factors, 

and  intra-hospital  nursing  units  (see  Appendix  I).  For  Category  inter-rater 

reliability.  Kappa  Statistic  was  used  with  significance  set  at  the  p  <  .01 
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level.  Analysis  of  variance  Intra-class  correlation  coefficient  (ICC)  was  used 

to  demonstrate  agreement  between  factors.  The  significance  level  was  set  at  p  < 

rt1  ,  .30 

.01  level. 

(6)  Pencil  and  paper  demographic  questionnaires  were  distributed  to  all 
professional  nurses  involved  with  the  Workload  Management  System.  In  addition, 
on  selected  nursing  units  the  charge  nurse  on  each  shift  for  three  consecutive 
days  was  requested  to  complete  a  Unit  Staffing  Evaluation  form  and  a  Nursing 
Care  Evaluation  form.  Staff  nurses  on  these  selected  units  were  also  requested 
to  complete  the  Nursing  Care  Evaluation  form. 

(7)  Data  from  the  questionnaires  were  compared  to  the  staffing  numbers 
obtained  from  the  daily  Workload  Management  System  Summary  Sheets  for  each  unit. 


A  Statistical  Analysis  Systems  Package  (SAS)  was  used  to  analyze  the 
Information.  Statistical  tests  performed  Included  Kappa  Statistic,  Intra-class 
correlations  (ICC)  and  Pearson  Product  Moment  r. 

6.  FINDINGS. 

a.  Instruments.  At  Naval  Hospital  Charleston,  37  patients  on  six  nursing 

units  representing  45  percent  of  the  Inpatient  census  were  classified  by  the 
nurse  expert  using  the  Patient  Classification  Critical  Indicator  Instrument.  To 
gather  subjective  information  from  the  charge  nurses  about  their  perceptions  of 
staffing  adequacy.  Unit  Staffing  Evaluation  forms  were  collected  for  a  total  of 

63  shifts.  To  evaluate  perceptions  of  care  given  during  these  shifts,  charge 

nurses  and  staff  nurses  also  completed  Nursing  Care  Evaluation  Forms. 

Demographic  data  to  characterize  the  respondents  included  information  regarding 
gender,  age,  rank,  and  title/position.  This  staff  questionnaire  also  elicited 
responses  regarding  length  of  time  to  classify  patients;  usefulness,  strengths, 
and  weaknesses  of  the  WMS;  and  suggestions  for  changes  in  the  Critical  Indicator 
Instrument.  A  total  of  64  demographic  staff  questionnaires  were  collected.  In 
addition,  four  supervisory  level  nurses  completed  the  Project  Manager's 
questionnaire. 

b.  Description  of  Subjects.  The  study  group  consisted  of  64 

registered  nurses.  The  group  was  comprised  of  15  males  and  49  females  with  70 

percent  being  under  34  years  of  age.  Among  the  nurses,  80  percent  were  staff 
nurses,  11  percent  were  charge  nurses  and  the  remaining  9  percent  worked  in  a 
variety  of  supervisory  roles.  All  subjects  had  experience  using  the  Patient 
Classification  Critical  Indicator  Instrument  but  only  10  nurses  had  seen  the 
graphs  and  daily  summary  sheets.  Over  94  percent  of  the  nurses  found  the 


Workload  Management  System  moderately  to  very  easy  to  use.  The  mean  time  to 
classify  a  patient  was  given  as  one  to  three  minutes  by  73  percent  of  the 
nurses. 

c.  Composite  Findings  on  the  Reliability  of  the  Workload  Management 
System.  Table  1  provides  a  comparison  of  the  patient  classification  category 
agreement  between  the  nurse  expert  and  charge  nurses  on  six  nursing  units.  The 
agreement  level  of  80  percent  for  Inter-rater  reliability  was  met.  Pearson 
product  moment  correlation  coefficient  (r)  demonstrated  a  high  correlation  of 
.93436.  To  rule  out  chance.  Kappa  Statistic  was  used.  The  Inter-rater 
reliability  was  significant  at  the  p  <  .001  level. 

To  analyze  the  Inter-rater  reliability  between  factors  on  the  Patient 
Classification  Critical  Indicators  Instrument,  the  Intra-class  correlation 
coefficient  (ICC)  was  used.  ICC  was  selected  over  Pearson  product  moment 
correlation  coefficient  (r)  to  estimate  reliability  because  Pearson  r  is  not 
sensitive  to  different  standards  between  raters,  so  that  raters  could  vary 
consistently  and  still  have  a  high  degree  of  reliability.  The  ICC  data  is 
presented  In  Table  2.  In  all  cases,  the  inter-rater  reliability  between  nurse 
expert  and  charge  nurses  for  the  factors  was  significant  at  the  p  <  .001  level. 
The  factors  with  the  lowest  correlation  were  teaching  and  emotional  support. 

This  finding  supports  the  historical  monthly  reliability  reports  submitted  by 
Naval  Hospital  Charleston. 

Intra-hospital  unit  reliability  using  the  Patient  Classification  Critical 
Indicators  Instrument  was  also  evaluated.  There  was  a  high  level  of  category 
agreement  (greater  than  80  percent)  between  units.  However,  because  of  the 
small  sample  size.  Kappa  Statistic  was  not  used  for  analysis.  After  information 
is  collected  from  the  five  other  study  sites,  it  will  be  possible  to  do 
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Table  1 


COMPARISON  OF  PATIENT  CLASSIFICATION  CATEGORY  AGREEMENT 
BETWEEN  NURSE  EXPERT  AND  CHARGE  NURSES  USING 
THE  CRITICAL  INDICATOR  INSTRUMENT 


Charge  Nurse  Rating 


Nurse  Expert 
Rating 


Category 

1 

Category 

2 

Category 

3 

Category 

4 

Category 

5 

Row 

Total 

Category 

1 

5 

1 

0 

0 

0 

6 

Category 

2 

1 

13 

1 

0 

0 

15 

Category 

3 

0 

1 

12 

0 

0 

13 

Category 

4 

0 

0 

0 

2 

0 

2 

Category 

5 

0 

! 

1 

0 

0 

0 

, 

, 

1 

1 

Column 

Total 

6 

15 

13 

12 

1 

37 

Percent  of  agreement  ■  33/37  or  89.21 
Kappa  Statistic  *  .892 
Standard  Deviation  of  Kappa  “  .075 
Z  score  «  11.25 

Category  agreement  was  significant  at  p  <  .001  using  Kappa  Statistic. 


Table  2 


INTER-RATER  RELIABILITIES  FOR  FACTORS  ON  THE  CRITICAL 
INDICATOR  INSTRUMENT  AS  ESTIMATED  BY  ICC 


r-% 


FACTOR 

Vital  Signs 
Monitoring 

Activities  of  Daily  Living 
Feeding 

Simple  Treatments 
Complex  Treatments 
Respiratory  Therapy 
Intravenous  Therapy 
Teaching 
Emotional 
Continuous  Care 


INTRA-CLASS  CORRELATION  (ICC) 


All  factors  were  statistically  significant  at  p  <  .001  using  the  f  test. 
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intra-hospital  reliability  testing  across  clinical  services. 

d.  Composite  Findings  on  the  Nurses1  Perceptions  of  Acceptability  and 
Satisfaction  With  the  Workload  Management  System.  Using  the  information 
obtained  from  the  Staff  Questionnaires  (n  «  64),  perceptions  of  satisfaction 
with  the  WMS  were  obtained.  Specifically,  perceptions  of  accuracy  in  reflecting 
level  of  care  required  by  patients,  usefulness  as  a  management  tool,  and 
satisfaction  with  the  WMS  as  a  whole  were  analyzed  (See  tables  3,  4,  5,  6).  An 
analysis  of  the  correlations  between  these  variables  and  the  roles  of  nursing 
staff  revealed  significant  differences  in  perceptions. 

Staff  nurses  (n  ■  38)  who  worked  on  the  clinical  units  rated  the  WMS  lower 
than  the  charge  nurses  or  supervisors  in  all  cases  (Table  3).  Of  this  group  few 
(n  »  3)  had  seen  the  entire  WMS  in  operation  which  included  the  staffing  summary 
sheets  and  monthly  graph  for  their  unit  (Table  4).  Rather,  their  perceptions 
appeared  to  be  based  solely  upon  completing  the  Patient  Classification  Worksheet. 

Charge  nurses  (n  *  7),  who  have  a  middle  management  role  in  the  hospital, 
perceived  the  system  somewhat  more  positively.  Of  these,  less  than  half  (n  »  3) 
Indicated  that  they  had  seen  the  monthly  reports  and  graphs  (Tables  4  and  5). 
Seventy-one  percent  of  the  charge  nurses  surveyed  thought  the  system  was  useful 
as  a  management  tool  and  accurately  reflected  the  nursing  care  required  (Tables 
3  and  6). 

Supervisory  level  nurses  (n  ■  6)  rated  the  WMS  at  the  highest  level  possible 
for  all  variables  (Tables  3,  4,  5,  and  6).  Except  for  the  Quality  Assurance 
Nurse,  the  supervisory  nurses  had  been  Involved  with  all  phases  of  the  WMS,  that 
Is,  patient  classification,  dally  adjustment  of  staffing  on  nursing  units  based 
upon  summary  reports,  and  review  of  monthly  staffing  reports  and  graphs  for  all 
units.  This  total  Involvement  with  the  WMS  may  have  contributed  to  the  very 
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PERCEPTIONS  OF  USEFULNESS  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 
AS  A  MANAGEMENT  TOOL  BY  NURSING  ROLE 


Nursing  Role 


Usefulness  of  WMS  ss 

a  Management  Tool 

♦ 

1 

Very  Useful/Useful 

Undecided/Not  Useful 

Row 

Total 

Staff 

Nuraee 

24 

(48%) 

26 

(52%) 

50 

(100%) 

Charge 

Nuraee 

5 

(71%) 

2 

(29%) 

Supervisor 

Level 

Nurses 

6 

(100%) 

0 

(0%) 

6 

(100%) 

Column 


35 


28 


Table  A 


SATISFACTION  WITH  WORKLOAD  MANAGEMENT  SYSTEM 
AMONG  NURSES  WHO  HAD  SEEN  MONTHLY  SUMMARY 
REPORTS  AND  GRAPHS  BY  NURSING  ROLE 


Controlling  Variable:  Saw  Reports  and  Graphs 


Satisfaction  With  WMS 


Nursing  Role 


Very  Satisfied/ 
Satisfied 

Neutral/ 

Dissatisfied 

Staff 

0 

3 

Nurses 

(OX) 

(100X) 

Charge 

1 

2 

Nurses 

C33.32) 

(66. 6X) 

Supervisor 

4 

0 

Level 

Nurses 

(100X) 

(OX) 

Column 

5 

5 

Table  5 


SATISFACTION  WITH  WORXLOAD  MANAGEMENT  SYSTEM 
AMONG  NURSES  WHO  HAD  NOT  SEEN  MONTHLY  SUMMARY 
REPORTS  AND  GRAPHS  BY  NURSING  ROLE 


Controlling  Variable:  Did  Not  See  Reports  and  Graphs 


Satisfaction  With  WMS 


Nursing  Role 


Very  Satisfied 
Satisfied 

Neutral/ 

Dissatisfied 

Very 

Dissatisfied 

Staff 

11 

23 

1 

Nurses 

(31Z) 

(66Z) 

(3Z) 

Charge 

1 

1 

0 

Nurses 

(50Z) 

(50Z) 

(OZ) 

Supervisor 

1 

0 

0 

Level 

Nurses 

(100Z) 

(OZ) 

(OZ) 

Column 

Total 


13  24  1 


Table  6 


PERCEPTIONS  OF  ACCURACY  OF  WORKLOAD  MANAGEMENT  SYSTEM 
IN  REFLECTING  LEVEL  OF  CARE  PATIENTS  REQUIRE 
BY  NURSING  ROLE 


Perception  of  Accuracy  of  WMS 


Nursing  Role 


Always /Usually 
Accurate 

Half  Time/Sometimes 
Accurate 

Never 

Accurate 

Staff 

26 

19 

5 

Nuraes 

(52%) 

(38%) 

(10%) 

Charge 

5 

2 

0 

Nurses 

(71%) 

(29%) 

(0%) 

Supervisor 

6 

0 

0 

Level 

Nurses 

(100%) 

(0%) 

(0%) 

Column 

37 

21 

5 

satisfied  responses  given  by  the  supervisory  level  nurses. 

The  analysis  of  the  nurses'  perceptions  of  the  major  strengths  and 
limitations  of  the  WMS  was  not  as  clear  (See  tables  7  and  8).  Usefulness  as  a 
management  tool  appears  to  be  the  greatest  strength  (n  ■  35).  However,  when 
major  strengths  were  compared  to  major  weaknesses  there  does  not  appear  to  be  a 
direct  relationship.  No  one  area  appeared  to  be  Identified  as  a  major  weakness. 
Rather  the  strengths  and  weaknesses  were  perceived  along  a  continuum  between  the 
two  extremes. 

e.  Correlation  Between  Charge  Nurses'  Perceptions  of  the  Quality  of  Nursing 

Care  Given,  Staffing  Adequacy,  and  Recommended  Staffing  Using  the  Workload 

Index.  From  findings  obtained  during  a  field  test  of  the  WMS  in  Apri 1-May  1983 

at  two  large  naval  hospitals.  It  was  believed  that  the  quality  of  nursing  care 

delivered  did  not  change  significantly  unless  a  nursing  unit  was  understaffed  by 

at  least  two  persons.  To  validate  this  finding  the  Workload  Index  was  set  using 
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the  criteria  of  plus  or  minus  two  persons.  Table  9  gives  a  breakdown  of  the 
Workload  Index  for  RN,  NRN,  and  total  combined  staff  over  63  shifts.  Of  special 
interest  is  the  fact  that  less  than  recommended  staffing  occurred  on  only  eight 
shifts  during  the  3  days  the  study  was  conducted  at  Naval  Hospital  Charleston. 
The  data  on  one  shift  could  not  be  included  because  the  charge  nurse  did  not 
complete  a  Unit  Staffing  Evaluation  questionnaire.  Therefore,  using  the  *2 
staff  member  criteria  only  seven  shifts  (11  percent)  had  less  than  the 
recommended  staff  while  eight  shifts  (13  percent)  had  more  than  the  recommended 
staff.  Due  to  the  small  size  of  these  samples,  the  correlation  between  quality 
of  nursing  care,  staffing  adequacy,  and  recommended  staffing  as  Indicated  by  the 
Workload  Index  were  not  evaluated  in  this  preliminary  study. 


Table  7 


MAJOR  STRENGTHS  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 


Variable 


Agree 

Frequency 


Percent 


Ease  of  Use 


18 


(28%) 


Comprehen  s ive 


16 


(25%) 


Accurately  reflects  workload 


11 


(17%) 


Takes  Little  Tine  TO  Do 


17 


(27%) 


Reliable 


(13%) 


Usefulness  as  Management  Tool 


35 


(55%) 


Table  8 


MAJOR  WEAKNESSES  OF  THE  WORKLOAD  MANAGEMENT  SYSTEM 


Variable  * 

Agree 

Frequency 

Percent 

Difficult  to  Use 

1 

(2%) 

Not  Comprehensive 

17 

(27%) 

Inaccurate  in  Reflecting  Workloac 

27 

(42%) 

Takes  Long  Time  to  do 

12 

(19%) 

Unreliable 

13 

(20%) 

Not  Useful  as  a  Management  Tool 


(6%) 


f.  Composite  Findings  on  Nurses1  Perceptions  of  Direct  and  Indirect  Nursing 
Care  6iven  Under  Various  Staffing  Levels.  Both  charge  nurses  and  staff  nurses 
rated  the  quality  of  direct  and  indirect  nursing  care  given  on  each  shift  (n  * 
113).  A  tabulation  of  frequencies  for  each  patient  classification  factor 
Indicated  that  the  nurses  perceived  direct  care  was  given  at  a  good/optimal 
level  at  least  80  percent  of  the  time,  except  for  teaching  and  emotional 
support.  Even  these  factors  were  placed  at  the  good/optimal  level  at  least  55 
percent  of  the  time.  An  analysis  of  the  Indirect  care  activities  also  revealed 

a  high  percentage  of  good/optimal  care  ratings.  Writing  patient  care  plans  and 

♦ 

making  patient  rounds  with  the  physician  received  the  lowest  ratings.  When 
comparing  charge  nurse  perceptions  (n  «  59)  to  staff  nurse  perceptions  (n  *  54) 
for  each  variable,  the  agreement  level  was  found  to  be  greater  than  80  percent. 


7.  DISCUSSION. 

The  results  of  this  preliminary  study  indicated  that  the  Workload  Management 
System  is  valid  and  reliable.  Correlations  between  nurses'  perceptions  and  the 
Workload  Index  were  not  possible  to  ascertain  due  to  the  small  sample  of  shifts 
that  had  less  than  or  more  than  recommended  staff.  Several  factors  need  to  be 
considered  when  collecting  data  at  future  test  sites. 

a.  The  Workload  Index  is  based  upon  staffing  numbers  and  mix  obtained  from 
the  dally  Workload  Management  Summary  Sheets  which  excludes  all  orientees 
(personnel  who  have  been  at  the  hospital  for  less  than  4  weeks).  Staff  nurses 
and  charge  nurses  were  not  specifically  informed  to  exclude  orientees  when 
rating  perceptions  of  care  given  on  staffing  adequacy.  A  review  of  the  number 
of  orientees  on  each  shift  at  NH  Charleston  revealed  15  orientees  (11  NRNs  on 
the  day  shift  and  4  NRNs  on  the  night  shift).  It  is  possible  that  the  presence 
of  orientees  on  a  nursing  unit  influenced  the  perceptions  of  good/optimal  care 


given  even  when  understaffed  by  two  persons. 

b.  Satisfaction  or  dissatisfaction  expressed  by  nurses  about  the  WMS  may  be 
tied  to  the  clinical  area  in  which  the  nurses  work.  A  review  of  written 
comments  on  the  demographic  staff  evaluation  questionnaire  Indicated  that  the 
nurses  in  the  labor  and  delivery  area  were  very  dissatisfied  with  the  WMS. 

Since  questions  were  not  asked  about  assigned  clinical  areas.  It  was  Impossible 
to  determine  If  nurses  In  specialty  settings  were  skewing  the  findings. 
Interestingly,  when  given  a  choice,  the  majority  (66X)  of  the  nurses  opted  to 
continue  using  the  system.  They  would  prefer  to  classify  once  a  day  or  more 
frequently  (95X)  and  do  that  classification  on  the  day  shift  (83X). 

c.  On  the  daily  Workload  Management  Summary  Sheets  completed  by  the  Nursing  Care 
Coordinators  (supervisors),  mistakes  were  occasionally  made  In  transcribing 

scores  from  the  Patient  Classification  Worksheets  and  in  determining  the  Nursing 
Care  Hours  required  on  a  unit.  In  rare  Instances,  nursing  personnel  were 
reassigned  by  supervisors  based  upon  erroneous  calculations  of  a  Work-load  Index 
for  a  nursing  unit. 

8.  CONCLUSION. 

The  Workload  Management  System  Patient  Classification  Critical  Indicators 
Instrument  was  found  to  be  very  reliable  for  factors,  and  for  categories  using 
Intra-class  correlations  and  Kappa  Statistic. 

Satisfaction  with  the  WMS  among  nurses  varied  according  to  nursing  role. 

Staff  nurses  were  neutral  or  dissatisfied  in  most  cases,  and  this  may  be  related 
to  the  fact  that  they  were  Involved  only  with  patient  classification  and  had  not 
seen  the  whole  system  in  operation.  Whether  or  not  all  staff  nurses  had 
reviewed  or  even  seen  the  Workload  Management  System  Educational  Workbook  is  a 
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pertinent  question  which  will  be  asked  at  future  test  sites.  A  review  of  the 
written  comments  Indicated  that  dissatisfaction  may  be  related  to  the  specialty 
area  In  which  a  nurse  works.  This  Information  will  be  collected  at  the  other 
five  test  sites.  Charge  nurses  rated  the  WMS  somewhat  higher,  and  those  who  had 
reviewed  the  stannary  reports  and  graphs  rated  It  very  high.  Supervisory  level 
nurses  rated  the  WMS  as  satisfactory  or  very  satisfactory. 

Because  less  than  recommended  staffing  occurred  on  only  seven  of  the  64 
shifts  during  the  time  of  the  study,  a  relationship  could  not  be  demonstrated 
between  charge  nurses'  perceptions  of  quality  of  care  given  or  staffing  adequacy 
with  the  Workload  Index.  Likewise,  perceptions  of  direct  and  Indirect  care 
activities  performed  under  various  staffing  configurations  could  not  be 
determined.  It  Is  anticipated  that  with  data  obtained  from  five  other  CONUS 
hospitals,  the  correlation  between  staffing  and  perceptions  of  quality  care 
given  can  be  evaluated. 


Naval  School  of  Health  Science  APPENDIX  A 

Research  Division 

WORKLOAD  MANAGEMENT  SYSTEM  FOR  NURSING 
STAFF  QUESTIONNAIRE 


1.  Gender 

1.  Male 

_ ______  2.  Fa  sale 

2.  Age 

_ 1.  20-2k 

2.  25-29 

3.  30-34 

_ 4.  35-39 

_ 5.  40-44 

_  6.  45-49 

7.  50-54 

_• _ 8.  55t 

3.  Rank 


1.  ENS 

7.  GS-7 

2.  LTJG 

8.  GS-8 

3.  LT 

_ 9.  OS-9 

4.  LCDS 

10.  GS-10 

5.  CDR 

11.  GS-11 

6.  CAPT 

4.  ntle/Poeition 

_  1.  Staff  Norse 

__ _ 2.  Charge  NOrse 

______  3.  Area  Coordinator 

_ 4.  Quality  Assurance  Coordinator 

_____  5.  Educational  Coordinator 
_____  6.  Director  or  Assistant  Director  of  Nursing 
7.  Other.  Snedfri 

5.  During  the  past  six  sooths,  hoar  frequently  did  you  use  the  Workload 

Nanagenent  Systea  to  classify  patients? 

_  1.  Each  shift 

_ 2.  Daily 

______  3.  Did  not  use 

6.  Hoar  long  does  it  take  you  to  classify  a  patient?  (on  the  average) 

______  Minutes  _ Seconds 

7.  Hoar  would  you  rate  the  ease  of  using  the  Workload  Manage  sent  System? 
_  1.  Very  Easy 

_____  2.  Easy 

______  3.  Moderately  Easy 

4.  Difficult 
______  5.  Very  Difficult 
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8.  Do  you  believe  the  categories  of  care  (I  to  VI)  determined  by  the  Workload 

Management  System  accurately  reflect  the  level  of  care  your  patients 
required? 

_____  1.  Always 

_  2.  Usually 

3.  Half  of  the  time 
_____  4.  Sometimes 
_____  5.  Sever 

9.  How  would  you  rate  the  usefulness  of  the  Workload  Management  System  as  a 

management  tool? 

1.  Very  Useful 

_ 2.  Useful 

_____  3.  Undecided 

_____  4.  Hot  useful 

_____  *>•  Hinderance  to  management 

10.  How  do  you  feel  about  the  Workload  Management  System  as  a  whole? 

_____  1.  Very  Satisfied 
_____  2.  Satisfied 

_  3.  Neutral 

"  4.  Dissatisfied 

_____  5*  Very  dissatisfied 

11.  What  do  you  see  as  the  major  strengths  of  this  Workload  Management  System? 

(You  may  select  more  than  one) 

_____  1.  Base  of  use 

_ 2.  Comprehensive.  (Content  is  complete) 

_____  3.  Accurately  re  flee  ts  jhe  workload 
_ 4.  Requires  minimum  time  to  complete 

_____  5-  Relaible.  (Same  results  obtained  from  one  staff  member  to  another) 

_ 6.  Useful  as  a  management  tool 

_  7.  Other.  Specify! 

"  8.  There  are  none 


12.  What  do  you  see  as  the  major  weaknesses  of  this  Workload  Management  System? 

(You  may  select  more  than  one) 

_____  1,  Difficult  to  use.  Complex. 

______  2.  Hot  comprehensive.  (Content  is  not  complete) 

_____  3.  Inaccurate.  (Does  not  reflect  the  workload) . 

_____  4.  Requires  too  much  time  to  complete. 

_____  5 •  Unreliable.  (Different  results  obtained  from  one  staff  member  to  another  \ 
6.  Hot  useful  as  a  management  tool 
_____  7*  Other.  Specify! 

_ 8.  There  are  none.  — — — — 


13.  Given  a  choice  would  yout 

_____  1.  Continue  to  use  this  system. 

______  2.  Develop  another  system. 

_____  3*  Use  another  existing  system.  Specify! 

______  4.  Use  no  classification  system. 

14.  How  often  do  you  think  patients  need  to  be  classified  in  order  to  accurately 

capture  your  workload? 

_____  1.  Every  shift. 

_____  2.  Once  every  24  hours 
_____  3-  Once  per  week 

4.  Other.  Specify! 


3 


15.  Should  you  ho  required  to  classify  your  patients  once  a  day*  which  shift 

do  you  believe  would  best  reflect  your  workload? 

1.  Days 

2.  Hveninga 

3.  Mights 

16.  Axe  there  a ay  significant  critical  iadioatocs  missing  that  you  believe 

should  be  included? 

1.  Xo 

_ 2.  Tee.  Specify! 


17..  Axe  there  any  significant  critical  indicators  that  you  believe  should  be 
deleted? 

_ 1.  Mo 

2.  Tee.  Specify i 


18.  Should  the  point  value  of  any  of  the  critical  indicators  be  changed? 
_ 1.  Ho 

_ 2.  Tee.  Specify  and  explaint 


19.  Could  the  Patient  Classification  Worksheet  be  better  designed  to  facilitate 
its  use? 

_ 1.  Ho 

_  2.  Tee.  Specify* 


20.  How  would  you  improve  the  Workload  Management  System? 


21.  General  Comments i 
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22.  Have  you  soon  the  Daily  Sussary  Sheets  and  the  Monthly  Suasary  Graphs 

cosparing  the  recossended  staffing  for  your  ward  baaed  on  the  Workload 
Managesent  System  to  your  actual  staffing  levels? 

1.  Mo 
_  2.  Yes 


If  your  answer  was  Yes  to  the  above  question  (#22) ,  please  complete  the  following 
by  circling  your  response  i 

HIGH  ADBQUASB  LOW 


1.  The  recossended  staff  levels  3  2  1 

for  the  day  shift  was 

2.  The  recossended  staffing  levels  3  2  1 

of  RM*8  for  the  day  shift  was 

3.  The  recossended  staffing  levels  3  2  1 

of  Para*s  for  the  day  shift  was 

4.  The  recossended  staffing  levels  3  2  1 

for  the  evening  shift  was 

5*  The  recossended  staffing  levels  3  2  1 

of  BN's  for  the  evening  shift  was 

6.  The  recossended  staffing  levels  3  2  1 

of  Bara's  for  the  evening  shift  was 

7*  The  recossended  staffing  levels  3  2  1 

for  the  night  shift  was 

8.  The  recossended  staffing  levels  3  2  1 

of  RN's  for  the  night  shift  was 

9.  The  recossended  staffing  levels  3  2  1 

of  Bara's  for  the  night  shift  was 
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APPENDIX  B 


WORKLOAD  MANAGEMENT  SYSTEM  FOR  NURSING 
PROJECT  MANAGERS 


To  be  completed  by  those  who  were  responsible  for  completing  the  reliability 
checks,  the  Daily  Summary  Sheets,  and  the  Monthly  Graphs. 

1.  How  long  did  it  take  you  to  do  the  reliability  testing,  on  the  average,  for 

nursing  unit?  __________  minutes. 

2.  How  often  do  you  believe  this  reliability  testing  is  needed? 

_______  1.  weekly 

______  2.  every  2  weeks 

________  3.  every  month 

_______  4.  every  other  month 

_______  5.  every  quarter 

______  6.  Other.  Specify i 

3.  At  your  Medical  Treatment  Facility,  who  would  be  the  best  one  to  conduct 

the  reliability  tests? 

_______  1.  Staff  Nurse 

_  2.  Charge  Nurse 

_  3.  Area  Coordinator 

_ 4.  Quality  Assurance  Coordinator 

______  5.  Educational  Coordinator 

_ 6.  Assistant  Director  of  Nursing 

_____  7.  Other.  Specify 

4.  What  is  your  perception  of  staff  acceptance  of  the  Workload  Management 

System? 

_____  1.  Very  Positive 

_  2.  Positive 

_____  3.  Neutral 
4.  Negative 

_____  5.  Very  Negative 

5.  How  would  you  rate  the  ease  of  using  the  Personnel  Requirements  Charts  to 

determine  the  recommended  staffing? 

______  1.  Very  Easy 

_____  2.  Easy 

_  3.  Moderately  Easy 

4.  Difficult 

5 .  Very  Difficult 

6.  Do  you  believe  the  Daily  Summary  Sheet  could  be  better  designed  to  facilitate 

its  use? 

_ 1.  No 

_____  2.  Tee.  Specify! 

7.  Do  you  believe  the  Monthly  Summary  Graph  could  be  better  designed  to  facil¬ 

itate  its  use? 

1.  No 
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Naval  School  of  Health  Sciences 
Research  Division 


WORKLOAD  MANAGEMENT  SYSTEM  FOR  NURSING 


UNIT  STAFFING  EVALUATION  FORM 


Hospital 

Date 

Unit _ 


Shifts  (  )  Days  07-1 5 

l  )  Evenings  15-23 
(  )  Nights  23-0? 


CHARGE  NURSES  s  Please  cos  plate  at  or  towards  the  end  of  the  shift. 


1. 


Many  factors  can  influence  scheduled  staffing  pattens  for  a  nursing  unit 
on  a  given  shift.  Please  evaluate  the  following  factors  as  they  apply 
to  the  shift  just  completed.  Circle  the  number  that  corresponds  to  your 


response* 

4  ■  Greater  than  usual 
3  ■  Usual 

2  a  Less  than  usual 
1  -  Not  applicable 


NUMBER  OF* 


a.  staff  scheduled* 


2 


1 


b.  RN's  schedules* 


2 


1 


c.  paraprofessionals  scheduled* 


2 


1 


d.  staff  sick,  called  in,  or  pulled  to 
another  ward* 

e.  admissions  on  this  shift 

f.  admissions  on  previous  shift 

g.  patients  transfered  in,  transfered  out, 
or  discharged* 


h.  post-operative  patients  (OR  today  or 
yesterday)* 


i.  patients  requiring  extensive  nursing 
care  times* 


2 

2 

2 


Z2 


2 

2 


1 

1 

1 

1 


j.  patients  requiring  special  treatments 
or  procedures  (IV* s,  suctioning, 
dressings,  diagnostic  tests)* 


k. 


patients  requiring  transport  or  escort 
by  nursing  staff* 
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4  3 

4  3 


2 


2 


Circle  the  fr**  that  corresponds  with  the  response  that  beet 
describes  the  shift  Just  completed. 

a.  In  general,  the  quality  of  naming  care  provided  the  patients 
daring  this  shift  east 

5  -  Optimal 

4  -  Good 

3  -  Adequate 

2  -  fair 

1  -  Poor 

b.  In  general,  the  staffing  far  this  shift  vast 

5  -  Optimal 

4  -  Good 

3  -  Adequate 

2  -  Fair 
1  -  Poor 

c.  Staffing  changes  were  needed i 

2  -  PO 

1  -  Tes 

d.  Additional  staff  was  needed* 

2  •  Jlo 
1  -  Tes 

Indicate  how  many  mam  staff  members  were  needed* 

_ RHs 

______  Famprofessionals 

_  MBTe/Ward  Clerks 

_ Other  (specify)  t 


e.  Less  staff  was  needed* 

2  -  Ro 
1  -  Tes 

Indicate  how  many  staff  members  were  not  needed* 
_ RHS 

_____  Paraprofessi  onals 
______  MBTs^Vard  Clerks 

___^  Other  (specify)  t  ________________ 


3.  Coa  plate  the  following  information* 


&.  Total  master  of  patients  in  aach  category  at  end  of  tho  ahiftt 

_______  Category  I 

_______  Category  II 

Category  III 

_ Category  IT 

_______  Category  T 

______  Category  VI 

b.  Abater  of  artal ami  ona  and  diaebazgaa  during  this  shifts 

________  Abater  of  adaisaions  and  tranafara  in  daring  this  shift. 

_ Abater  of  discharges  and  transfers  out  during  this  shift. 

e.  Aoater  and  air  of  personnel  that  worked  on  your  unit  this  shifts 
PBBSCMAEL _ NUMBER _ HOURS  WORKED 


orientees 


Paraprofessionals 
Ward  Clerks 


Others  (specify)  i 


TOTALS* 


00  ACT  WRITS  IA  THIS  SPACE.  TO  BE  COMPLETED  BY  THE  RESEARCHER 


orit] 


OPEN  1 - I  NURSERY  □ 

BOOMS  CH  LIC8T  1  I 


CLASS 


PATIENTS 


D  |e  |n 


CARE  HOURS 


D  1  S  1  A  ACTUAL 


ACUTE  □ 

ICU  □ 


RA 


PARA 


TOTAL 


APPENDIX  D 


NAVAL  SCHOOL  OF  HEALTH  SCIENCES 
RESEARCH  DIVISION 

WORKLOAD  MANAGEMENT  SYSTEM  FOR  NURSING 
NURSING  CARE  EVALUATION  FORM 


Hospital: 

Date: _ 

Unit: 


Shift:  (  )Days  07-15 

(  ) Evenings  15-23 

(  )Nighta  23-07 


CHARGE  NURSE:  Please  complete  at  or  towards  the  end  of  the  shift 


1.  Evaluate  to  the  best  of  your  knowledge  the  quality  of  nursing  care  provided 
the  patients  during  this  shift.  Using  the  following  scale,  circle  your  response. 


5-Optlmal  care 

4 -Good  care 

3-Adequate  care 

2-Fair  care 

1-Poor  care 

0-Not  applicable 

DIRECT  CARE: 

I 

Good  care 

Adequate  care 

Fair  care 

Poor  care 

Not  applicable 

THE  PERFORMANCE  AND  ASSESSMENT  OF: 

a.  vital  signs; 

5 

4 

3 

2 

1 

1 

b. 

monitoring  activities  (I&O;  circulation 

5 

4 

3 

2 

1 

0 

c. 

fundus  and  neuro  checks;  cardiac,  apnea, 
temperature  &  pressure  monitoring); 

activities  of  dally  living  (baths. 

5 

4 

3 

2 

1 

0 

d. 

weights,  toileting,  positioning  & 
routine  patient  assessment) ; 

nutritional  activities  (tube  feedings. 

5 

4 

3 

2 

1 

0 

e. 

bottle  feedings,  TPN) ; 

treatments,  procedures,  and  medication 

5 

4 

3 

2 

1 

0 

f. 

administration  (dressings,  ambulation 
of  patients,  assisting  the  MD) ; 

respiratory  treatments  (02,  IPPB 

5 

4 

3 

2 

1 

0 

8* 

incentive  spirometer,  chest  PT, 
trachea  care,  suctioning); 

intravenous  therapy  (dressing  changes. 

5 

4 

3 

2 

1 

0 

h. 

IV  medication,  blood  products); 

teaching  (Pre-op,  admission,  special); 

5 

4 

3 

2 

1 

0 

i. 

patient  and  family  emotional  support 
(modification  of  lifestyle,  sensory 
deprivation) . 

46 

Not  applicable 
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NAVAL  SCHOOL  OF  HEALTH  SCIENCES 
RESEARCH  DIVISION 

WORKLOAD  MANAGEMENT  SYSTEM  FOR  NURSING 
NURSING  CARE  EVALUATION  FORM 


Hospital: 

Date: _ 

Unit: _ 


Shift:  (  )Days  07-15 

(  ) Evenings  15-23 

(  ) Nights  23-07 


STAFF  NURSES:  Please  complete  at  or  towards  the  end  of  the  shift 


1.  Evaluate  to  the  best  of  your  knowledge  the  quality  of  nursing  care  provided 
the  patients  during  this  shift.  Using  the  following  scale,  circle  your  response. 


5-Optlaal  care 

4-Good  care 

3- Adequate  care 

2-Fair  care 

1-Poor  care 

0-Not  applicable 

\ 

DIRECT  CARE: 

Optimal  care 

Good  care 

Adequate  care 

Fair  care 

THE  PERFORMANCE  AND  ASSESSMENT  OF: 

a. 

vital  signs; 

5 

4 

3 

2 

b. 

monitoring  activities  (I&O;  circulation 
fundus  and  neuro  checks;  cardiac,'  apnea, 
temperature  &  pressure  monitoring); 

5 

4 

3 

2 

c. 

activities  of  daily  living  (baths, 
weights,  toileting,  positioning  & 
routine  patient  assessment); 

5 

4 

3 

2 

d. 

nutritional  activities  (tube  feedings, 
bottle  feedings,  TPN) ; 

5 

4 

3 

2 

e. 

treatments,  procedures,  and  medication 
administration  (dressings,  ambulation 
of  patients,  assisting  the  MD) ; 

5 

4 

3 

2 

f. 

respiratory  treatments  (02,  IPPB 
incentive  spirometer,  chest  PT, 
trachea  care,  suctioning); 

5 

4 

3 

2 

g- 

intravenous  therapy  (dressing  changes, 

IV  medication,  blood  products); 

5 

4 

3 

2 

h. 

teaching  (Pre-op,  admission,  special); 

5 

4 

3 

2 

1. 

patient  and  family  emotional  support 
(modification  of  lifestyle,  sensory 
deprivation).  4g  . 

M 

<0 

O 

M 

O 

O 

d. 


1 


0 


1 


0 


1 


1 


0 


0 


1 


0 


1 


0 


1 


0 


1 


0 


I 


Not  applicable 


Mm 

Mo»ptC<l  U1C 


WORKLOAD  MANAGEMENT  SUMMARY  SHEET 
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OPEN  O  ROOMS 

LIGHT  Q  NURSERY 


ACUTE  CD 
ICU  £7 


ADMISSIONS 


ADMISSIONS 


OPEN  CD 
LIGHT  CD  - 


dMk«r  •( 
CI<m  PiCinti 


ROOMS  CD 

YO 


T«tal 


Orintaai 


ADMISSIONS 
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PAT1INT  CLASSIFICATION  CRITICAL  INDICATORS 
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Ift  Dana*  pat-** 

anmnamfl 
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(41  PAP/PA  Mftt  ntMf  42k  a  1 12 
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114)  Turaai  fncn  (2  nftl  to  turn  q2W 
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12)  MaiiMaau  MM*  i  1  la 
II 2)  Hfawksawit  MtM  «4A  a  i  8 
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it  4  a  an 
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i  a  caoa  44k 

I  a  atuiHt  Hail 
4)  :PP8  a  attaint  48k  a  •  4 
81  (PPS  a  mtiattt  t4k  a  1  8 
•  ■81  Cioaa  ttft  a  mat  tan 


(4)  A  VO 
(41 


181  SMftt  lektaa  MMt  4M  laait 
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Timas 

linSa 


■41 
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■V  TH8WAPY 

12)  MtactDai  48k  a  1  3 

13)  Mtactkai  46k  a « 4 

14)  M  tact  cat  44k  a  I  8 
(41  BM  pnM 

TMactMkta  am  — potumal  aumowr 
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(4)  PtSSSIaSy  a(*a«  (U.  auiaty.  canal.  Makratt.  nc 
(41  MtCHktuai  ft  fttaftt  (U.  tea  antkeat.  toar  «agt. 
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(8)  Stntay  dtsnttnn  (U  rtiattt,  4tt(.  Mnt.  laiftttt 
Mtiaft  tyt  tticka,  no  hand,  caaktuvt.  tic.) 

(10)  Mauman  past*  fa  1 
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NOTES t  I,  Double  points  for  any  treatment/procedure  that  requires  2  nursing  staff  members 

2.  Adjust  points  to  accomodate  frequency,  l.e.  IPPB  q2h  ■  12  points. 

3.  Admissions  and  day  of  surgery  are  automatically  st  least  a  Category  It. 

A.  Count  only  thoee  procedures  performed  by  the  nursing  staff. 
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PATIENT  CLASSIFICATION  RELIABILITY  TESTING  INSTRUMENT 


Ward/Unit  Date  _ 

Census _  Signed 


SAMPLE  SELECTION  (jLgjf&Si 

census 


) 


i  agreement 


Category  1 

Category  2 

Category  3 

Category  L 

Category  5 

Category  6 

Total 

AGREEMENT  BY  CATEGORY  checked)  *  agreement 


Category  1 

Category  2 

Category  3 

Category  U 

Category  5 

Category  6 

.  Total 

AGREEMENT  BY  FACTORS  (L&&SS&-) 

#  checked 


£  agreement 
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WORKLOAD  MANAGEMENT  SYSTEM  EVALUATION  STUDY 


•  FOR. OFFICIAL  USE  ONLY  * 
DISCLOSURE  STATEMENT 


The  purpose  of  this  study  is  to  evaluate  the  validity  and  reliability 
of  the  Workload  Management  System  (WMS)  for  Nursing.  In  agreeing  to 
participate  in  this  study,  you  are  being  asked  to  complete  a  questionnaire 
concerning: 

(1)  Demographic  information  and  an  overview  of  your  assessment  of  the  WMS. 

(2)  An  assessment  of  the  staffing  assigned  to  your  unit. 

i 

(3)  An  assessment  of  the  nursing  care  given  to  patients  on  your  shift. 


(4)  For  WMS  project  managers,  an  assessment  of  the  forms  used  to  collect 
workload  to  determine  staffing. 

You  should  understand  that  ALL  questionnaire  answers  and  other  data  you 
provide  WILL  BE  TREATED  AS  STRICTLY  CONFIDENTIAL,  and  will  be  used  for  RESEARCH 
PURPOSES  ONLY.  All  questionnaires  will  be  returned  directly  to  the  Research 
Division,  Naval  School  of  Health  Sciences,  Bethesda,  MD,  where  they  will  be  kept. 
The  results  of  this  study  will  be  tabulated  and  reported  in  summary  form  only,  and 
it  will  NOT  be  possible  to  identify  any  single  individual  in  the  report. 


You  should  also  understand  that  (a)  your  participation  in  this  study  is 
encouraged,  but  voluntary,  (b)  that  the  results  of  this  study  will  be  published 
in  a  technical  report  which  will  be  pade  available  to  your  hospital,  and  (c)  there 
are  no  immediate  benefits  to  you  personally  for  participating  in  this  study, 
although  your  feedback  now  may  be  instrumental  in  making  constructive  modifications 
to  the  system. 


Finally,  you  should  also  understand  that  if  you  have  any  questions  regarding 
this  study  at  any  time  in  the  future,  you  may  contact  the  following  individual 
who  will  assist  you. 


CDR  KAREN  A.  RIEDER,  NC,  USN 
Research  Department 
Naval  School  of  Health  Sciences 
Bethesda,  Maryland  20814 

Phone:  (202:>  295-1467 


THANK  YOU  FOR  YOUR  COOPERATION 
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^  To  evaluaCe  reliability,  37  patients  on  six  nursing  units  in  the  first 
test  hospital  were  classified  by  the  nurse  expert  using  the  patient  classifi¬ 
cation  Critical  Indicator  tool.  To  gather  subjective  information  from  charge 
nurses  about  their  perceptions  of  staffing  adequacy.  Unit  Staffing  Evaluation 
tools  were  completed  for  a  total  of  63  shifts.  Perceptions  of  care  given 
during  these  shifts  were  obtained  from  charge  nurses  and  staff  nurses  using- 
the  Nursing  Care  Evaluation  form.  Demographic  data  to  characterize  the 
respondents  and  to  evaluate  user  acceptance  were  collected  using  a  staff 
questionnaire  (n  -  64). 

Inter-rater  reliability  was  examined  using  Kappa  Statistic  and  Intra¬ 
class  correlations.  The  relationship  between  recommended  staff  numbers  and 
nurses  perceptions  of  staffing  adequacy  was  measured  using  correlational 
statistics.  Descriptive  statistics  were. utilized^ to  determine  user  accept¬ 


ability  of  the  system. 


CONCLUSIONS : 
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sinter-rater  reliability  between  staff  nurses  and  the  nurse  investigator 
exceeded  the  80Z  agreement  level  with  a  Pearson  correlatiop  of  .93. ^Cate¬ 
gory  agreement  was  significant  at  p  7 .Q01  using  Kappa  Statistic.  Inter-rater 
reliability  for  factors  within  the  patient  classification  critical  indicator 
instrument  ranged  from  .763  to  .997  using  Intra-class  correlation.  All  factors 
were  significant  at  p  7.001  using  the  f  test. 

Using  information  from  the  staff  questionnaires,  the  system  was  rated  as 
moderately  or  very  easy  to  use  by  94Z  of  the  nurses  with  a  mean  time  to  classify 
patients  given  as  one  to  three  minutes  by  73Z  of  the  staff.  The  system  was 
seen  as  being  usually  or  always  accurate  by  59Z,  useful  or  very  useful  by  56Z, 
and  73Z  were  satisfied  or  neutral  toward  the  system.  The  major  strength  of 
the  system  was  its  usefulness  as  a  management  tool;  the  major  weakness  identi¬ 
fied  was  its  inaccuracy  in  reflecting  workload  in  some  specialty  areas. 

Because  only  eight  of  the  64  shifts  had  less  than  recommended  staffing  (-2 
staff),  the  correlation  between  nurses  perceptions  of  quality  care  given, 
staffing  adequacy,  and  recommended  staffing  could  not  be  ascertained. 

RECOMMENDATION : 

Results  of  this  study  be  combined  with  the  findings  from  the  other  five 
test  hospitals.  Final  modifications  of  the  system  be  made  based  on  these 
findings. 
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